Skip to content

harden localedata loading with restricted unpickler - #1272

Closed
uwezkhan wants to merge 1 commit into
python-babel:masterfrom
uwezkhan:safe-localedata-unpickler
Closed

uwezkhan wants to merge 1 commit into
python-babel:masterfrom
uwezkhan:safe-localedata-unpickler

Conversation

@uwezkhan

Copy link
Copy Markdown

this replaces the direct pickle.load() usage in babel.localedata with a restricted unpickler.

right now locale data files are loaded using pickle without restricting what objects can be created during deserialization. this change adds a _SafeUnpickler which only allows the small set of classes and builtin types that babel actually needs for locale data loading.

also added security tests to make sure:

normal locale data still loads correctly
malicious pickle payloads are rejected
unsafe globals cannot be loaded during deserialization

the existing .dat files continue to work and no changes are needed for the locale generation process.

this is mainly a defense in depth hardening change to make locale data loading safer against malicious or corrupted pickle data.

@uwezkhan

uwezkhan commented Jul 1, 2026

Copy link
Copy Markdown
Author

any update ?

@akx akx left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR, but I'm not sure it's necessary or complete.

  • Did you try this with the current Babel data?
  • Can you construct a scenario where def load() can load an arbitrary attacker-constructed pickle, or how an attacker could modify a pickle in the data directory to be malicious (without having permissions to similarly modify babel/localedata.py)?

@uwezkhan

uwezkhan commented Oct 5, 2026

Copy link
Copy Markdown
Author

It doesn't hold up on either point.

Current data. The PR only tested hand-built pickles, not the shipped files. Against a fresh CLDR 47 import it breaks:

  • Before (plain pickle.load): localedata.load('en_US') works.
  • After (this PR): UnpicklingError: Global babel.dates.DateTimePattern is forbidden.

The allow-list is missing babel.dates.DateTimePattern and babel.numbers.NumberPattern. root fails to load, so every locale fails with it, and the suite goes to 6952 failed, 530 passed. The Decimal entry isn't referenced by any .dat file, and the global.dat load in core.py isn't touched. The line in the description saying the existing .dat files keep working is wrong.

Scenario. I can't construct one. load() reduces the name with os.path.basename() and only opens <package>/locale-data/<name>.dat, so the attacker needs write access to the package directory, which is the same access needed to edit localedata.py. That matches T-3 and E-2 in SECURITY.md.

Tradeoff. A corrected allow-list would have to track whatever import_cldr.py pickles, and it still wouldn't be a boundary: PluralRule has to be on it, and it compiles its unpickled state to Python (E-4). That's ongoing maintenance for no gain under the stated threat model, so I don't have a case for merging this. Feel free to close it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants